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CORRELATION  STUDIES  ON  A  PROTOTYPE  COLOR-MEASUREMENT  SYSTEM 


INTRODUCTION 


Overview  of  Project 

One  of  the  problems  the  Army  faces  in  the  procurement  of  the  millions  of  yards  of 
textile  fabrics  it  needs  each  year  is  the  judging  of  color  acceptability.  At  present,  the  color 
is  judged  by  comparing  a  specimen  visually  to  a  standard  and  several,  usually  eight,  samples 
that  define  the  thin  and  full  limits.  If  the  color  is  within  the  bounds  of  these  limits,  it  is 
acceptable.  Otherwise  it  fails.  This  procedure  is  not  objective  and  can  lead  to  disagreements 
in  the  case  of  a  borderline  sample.  It  is  desirable  to  have  a  method  of  measuring  the  samples 
in  an  objective  and  repeatable  way  to  determine  whether  or  not  they  meet  the  Army's 
requirements. 

In  recent  years,  color  measuring  instruments  have  become  faster  and  more  convenient 
to  use,  with  good  precision  and  reproducibility.  They  are  all  interfaced  to  computers  to  do 
any  necessary  calculations  and  are  available  at  a  moderate  cost.  Because  of  this,  the  Army 
has  initiated  a  program  to  develop  and  validate  an  objective  color  measurement  system  for 
the  determination  of  textile  color  acceptability.*  This  program  consists  of  four  phases  that 
are  summarized  below. 

Phase  I  was  a  survey  of  commercial  equipment.*  Three  new,  relatively  inexpensive 
spectrophotometers  were  studied  to  see  if  they  could  meet  the  Army's  requirements  for 
reproducibility  and  accuracy.  All  three  instruments  tested,  the  Macbeth  MS— 2000,  the  Diano 
Match  Scan  and  the  Hunter  D54— P,  were  found  to  be  suitable  with  modifications. 

Phase  11  was  the  design  of  the  system.  Here,  the  necessary  modifications  were  determined 
and  a  detailed  specification  for  procurement  of  a  prototype  system  written.  Also  includ*"'  in 
this  phase  was  the  development  of  a  method  for  determining  the  acceptability  parameters  the 
computer  would  need  to  determine  if  a  sample  passed*  and  a  fail-ixife  method  of  calibration 
that  would  allow  the  instrument  to  be  run  only  if  it  is  operating  correctly.^ 

'A.O.  Ramsiey,  T.R.  Commerford,  and  L.B.  Hepfinger.  Objective  Color  Measuring  System. 
NATICK/TR— 83/CX)5,  US  Army  Natick  Research  and  Development  Laboratories,  Natick,  MA 
01760,  September  1982,  AD  A124  505. 

*F.W.  Billmeyer,  Jr.  and  P.J.  Alessi.  Assessment  of  color-measuring  instruments  for  objective 
textile  acceptability  judgement.  NATICK/TR— 79/044,  US  Army  Research  and  Development 
Command,  Natick,  MA  01760,  March  1979,  AD  A081  231. 

*E.  Allen  and  B.  Yuhas.  Investigation  to  define  acceptability  tolerance  ranges  in  various  regions 
of  color  space.  NATICK/TR— 80/036,  US  Army  Natick  Research  and  Development  Laboratories, 
Natick,  MA  01760,  September  1980,  AD  A094  163. 

^F.J.  Simon  and  J.H.  Lubar.  Standardization  procedure  for  two  instruments  for  color 
measurement.  NATICK/TR— 82/024,  US  Army  Natick  Research  and  Development  Laboratories, 
Natick,  MA  01760,  September  1981,  AD  A116  350. 


In  Phase  III,  the  prototype  instrument  was  procured  and  tested.  This  report  details  part 
of  the  validation  studies  conducted  on  the  prototype  system. 

The  system  will  be  tried  out  on  an  actual  procurement  in  Phase  IV.  Acceptability  will 
be  judged  on  the  trial's  results  but  will  be  checked  by  visual  observation. 

Phase  III.  The  NLABS— DPSC  Prototype  Color  Measurement  System 

The  prototype  system  that  was  procured  through  competitive  bidding  consists  of  two 
Applied  Color  Systems  (ACS)  Spectro  Sensors,  one  instrument  located  at  Natick  Labs  (NLABS) 
and  another  at  the  Defense  Personnel  Support  Center  (DPSC)  in  Philadelphia.  The  computers 
that  run  the  two  instruments  are  connected  to  each  other  over  telephone  lines.  Because 
specifications  on  the  color  of  textiles  to  be  procured  and  tested  at  DPSC  are  determined  at 
NLABS,  the  two  instruments  must  give  the  same  results  when  they  measure  a  given  sample. 
They  must  also  agree  with  themselves  and  each  other  over  short  and  long  periods  of  time, 
so  that  measurements  made  next  year  will  be  consistent  with  measurements  made  this  year. 

EXPERIMENTAL  PROCEDURE 

A  set  of  samples  were  measured  on  the  NLABS  instrument  and  then  taken  to  DPSC  to 
be  measured.  The  samples  were: 

1.  Fifteen  4"  x  4"  porcelain  enamel  tiles  consisting  of  five  standards  (two  tan,  two 
green  and  one  blue)  with  a  full  and  thin  limit  sample  for  each  standard; 

2.  Twenty  4"  x  10"  Nyco  twill  fabric  swatches,  10  tan  and  10  green; 

3.  A  pair  of  tan  polyester  gelcoat  plaques. 

Figures  1  through  3  show  plots  of  these  samples. 

The  tile  and  fabric  samples  were  measured  once  a  day  for  10  days  for  the  short-term 
test  and  the  tiles  once  a  week  for  10  weeks  for  the  long-term  test.  The  fabric  samples  were 
measured  twice  a  week  for  five  weeks  for  their  long-term  test.  The  tan  color  difference  pair 
was  measured  on  10  separate  days  for  three  weeks. 

The  tiles  were  marked  on  the  back  with  a  circle  to  ensure  that  the  same  area  was  measured 
each  time  and  an  arrow  indicated  the  up  direction.  Three  readings  were  taken  without  moving 
the  sample  and  an  average  taken.  The  textile  samples  were  marked  with  four  circles  across 
their  length,  with  arrows  showing  the  up  direction  so  the  twill  lines  were  positioned  in  a 
reproducible  manner.  The  samples  were  backed  with  a  grey  tile  during  measurement  and  the 
four  readings  were  averaged.  Each  half  of  the  tan  color  difference  pair  also  had  a  measurement 
circle  marked  on  the  back  and  each  was  measured  three  times  and  averaged.  A  grey  tile  backed 
up  these  samples  also,  because  they  were  slightly  translucent. 
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The  samples  were  measured  with  a  daylight  quality  source  in  the  specular  included,  large 
area  of  view  mode.  The  10°  1964  CIE  Supplementary  Standard  Observer  and  CIE  Standard 
llluminant  D65  were  used  for  all  tristimulus  value  calculations.  The  CIE  1976  L*a*b*  (CIELAB) 
color  space  was  used  for  all  color  difference  calculations. 

RESULTS  AND  DISCUSSION 
Short-Term  Comparison  of  the  Two  Instruments 

Table  la,  1b,  and  1c  contain  the  results  of  DPSC's  short-term  study  compared  to  those 
of  NLABS.  In  this  and  following  tables,  the  standard  deviations  of  the  tristimulus  values  and 
MCDM's  (mean  color  difference  from  the  mean  of  a  set  of  measurements)^  are  given  below 
the  values.  For  a  given  sample,  the  mean  value  of  X  for  NLABS  is  compared  to  the  mean 
value  of  X  for  DPSC.  If  the  X  values  are  between  one  and  two  standard  deviations,  they 
are  marked  with  one  asterisk,  while  two  asterisks  indicate  the  means  are  more  than  two  standard 
deviations  apart.  No  asterisk  indicates  the  means  are  within  one  standard  deviation.  The 
Y  and  Z  values  are  treated  the  same  way.  The  color  difference,  AE*,  is  the  CIELAB  color 
difference  between  the  two  means. 

From  Table  la,  1b,  and  1c  it  can  be  seen  that  the  MCDM's  for  DPSC  are  about  the 
same  as  those  for  NLABS,  but  the  tristimulus  values  are  generally  higher  than  Natick's.  Many 
of  the  measurements  on  the  green  fabric  samples  do  not  fall  within  one  standard  deviation 
of  each  other,  but  are  usually  within  two  standard  deviations.  All  of  the  tan  readings  are 
within  two  standard  deviations  of  each  other,  and  about  half  are  within  one.  While  this  makes 
the  means  significantly  different  statistically,  the  largest  AE*  that  results  is  only  0.13  CIELAB 
units.  The  largest  AE*  that  is  observed  in  data  that  is  within  one  standard  deviation  is  0.18 
CIELAB  units.  Because  most  of  the  allowable  color  differences  that  will  be  encountered  in 
practice  will  be  much  larger  than  this,  it  should  not  cause  a  problem.  The  color  differences 
between  the  NLABS  and  DPSC  means  are  generally  larger  than  the  MCDM's  for  each  instrument's 
short-term  results,  ranging  from  0.03  to  0.18  CIELAB  units,  with  an  average  of  0.09  CIELAB 
units.  These  results  are  consistent  with  an  instrument  comparison  done  on  the  Spectro  Sensor 
several  years  ago  by  ACS.* 

Long-Term  Comparison  of  the  Two  Instruments 

Table  2a,  2b,  and  2c  contain  the  long-term  results  for  the  two  instruments.  All  of  the 
tile  readings  are  within  two  standard  deviations  of  each  other  and  color  differences  between 
the  two  sets  range  from  0.07  to  0.18.  The  AE*'s  in  this  set  are  slightly  higher  than  for 


*  R.  Stanziola,  B.  Momiroff,  and  H.  Hemmendinger.  The  Spectro  Sensor  -  a  new  generation 
spectrophotometer.  Color  Res.  Appl.,  4,  157—163,  1979. 


the  short-term  study,  but  are  still  small  in  practical  terms.  The  fabric  samples,  which  were 
only  measured  for  five  weeks,  show  smaller  color  differences  between  the  two  sets  of  data 
than  the  tile  samples  do,  but,  again,  the  majority  of  the  green  measurements  show  a  statistically 
significant  difference  between  their  means.  This,  however,  results  in  an  acceptably  small  color 
difference,  as  in  the  short-term  results. 

Short-Term  vs.  Long-Term  Measurements  on  each  Instrument 

Both  instruments  exhibit  good  agreement  with  themselves  over  a  short  and  long  time. 
Color  differences  between  the  sets  range  from  0.00  to  0.16  CIELAB  units,  and  all  but  one 
pair  of  tristimulus  values  for  each  data  set  are  within  one  standard  deviation  of  each  other. 
These  results  are  given  in  Tables  3a,  3b,  and  3c  and  4a,  4b,  and  4c. 

Measurement  of  a  Tan  Color-Difference  Pair  on  each  Instrument 

The  color  differences  found  for  the  pair  of  tan  polyester  gel  coat  plaques  are  listed  in 
Table  5.  The  two  instruments  show  excellent  agreement  and  the  means  of  the  ten  measurements 
are  within  one  standard  deviation  of  each  other. 

Measurements  in  the  infrared  Region 

Because  some  Army  fabrics  have  requirements  for  reflectance  in  the  infrared,  this  factor 
was  also  investigated.  From  the  data  on  the  tan  and  green  fabric  samples,  integrated  values 
in  the  infrared  region,  up  to  900  nm,  were  obtained.  This  integration,®  which  is  similar  to 
the  calculations  for  Y  and  L*,  in  the  visible  region,  is  based  on  the  sensitivity  of  the  starlight 
scope,  a  nighttime  surveillance  device.  The  values  Ns  and  Ls,  which  are  the  nighttime  equivalents 
to  Y  and  L*  respectively,  are  calculated  for  two  different  illuminants:  illumination  representing 
a  moonlit  sky  and  a  moonless  night.  The  results  are  shown  in  Tables  6a  and  6b  through  9a 
and  9b. 

The  intrainstrument  comparisons  for  short-  and  long-term  time  periods  in  Tables  8  and 
9  show  excellent  agreement  with  only  one  reading  being  outside  of  the  one  standard  deviation 
limit.  The  interinstrument  comparison  in  Tables  6a  and  6b  and  7a  and  7b  is  not  as  good: 
most  readings  are  more  than  one  standard  deviation  from  each  other  and  about  half  are  more 
than  two  standard  deviations  apart.  As  with  the  visual  color  differences,  these  results  show 
a  significant  statistical  difference.  These  differences,  however,  are  considered  small  enough 
in  comparison  to  specification  requirements  to  be  acceptable. 


®A.O.  Ramsiey  and  W.G.Y.  Yeomans.  Psychophysics  of  Modern  Camouflage.  Presented  at 
the  Army  Science  Conference,  US  Military  Academy,  West  Point,  NY,  15—18  June  1982. 
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CONCLUSIONS 


Tables  10a  through  lOd  contain  a  summary  of  the  average  MCDM's  and  color  differences 
for  the  samples  and  their  standard  deviations  in  the  visible  region.  On  the  whole,  the  two 
instruments  show  the  same  degree  of  repeatability  in  the  visible  and  near-infrared  over  the 
short  and  long  periods  of  time  studied,  and  show  good  agreement  between  themselves.  As 
long  as  color  differences  greater  than  0.2  Cl E LAB  units  are  being  measured,  the  agreement 
between  the  two  instruments  should  be  adequate.  Because  the  color  difference  between  most 
standards  and  their  limit  samples  is  much  larger  than  this,  the  new  system  is  suitable  for  the 
Army's  needs.  Long-term  studies  on  both  instruments  should  continue  in  order  to  keep  a 
record  of  the  performance  of  the  two  instruments  and  ascertain  that  they  stay  in  close  agreement 
with  each  other  over  a  period  of  years. 
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Table  la.  Comparison  of  Short-Term  Repeatability  for  the  Tile  Samples  between  NLABS  and  DPSC: 

Standard  Deviation  of  Tristimulus  Values  (X,Y,Z) 


Table  1b.  Comparison  of  Short-Term  Repeatability  for  the  Tan  Fabric  Samples  between  NLABS  and  DPSC 
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Table  3a.  Comparison  of  Short-  and  Long-Term  Repeatability  for  the  Tile  Samples  on  the  NLABS 
Spectro  Sensor:  Standard  Deviation  of  Tristimulus  Values  (X,Y,Z) 
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Table  3a.  Comparison  of  Short-  and  Long-Term  Repeatability  for  the  Tile  Samples  on  the  NLABS 
Spectro  Sensor:  Standard  Deviation  of  Tristimulus  Values  (X,Y,Z)  (cont'd) 


Table  3b.  Comparison  of  Short-  and  Long-Term  Repeatability  for  the  Tan  Fabric  Samples  on  the  NLABS  Spectro  Sensor 


Table  3c.  Comparison  of  Short-  and  Long-Term  Repeatability  for  the  Green  Fabric 

Samples  on  the  NLABS  Spectro  Sensor 


Table  4a.  Comparison  of  Short-  and  Long-Term  Repeatability  for  the  Tile  Samples  on  the  DPSC 
Spectro  Sensor:  Standard  Deviation  of  Tristimulus  Values  (X,Y,Z) 
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Table  4a.  Comparison  of  Short-  and  Long-Term  Repeatability  for  the  Tile  Samples  on  the  DPSC 
Spectro  Sensor:  Standard  Deviation  of  Tristimulus  Values  (X,Y,Z)  (oont'd) 


Comparison  of  Short-  and  Long-Term  Repeatability  for  the  Tan  Fabric 
Samples  on  the  DPSC  Spectro  Sensor 
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Comparison  of  Short-  and  Long-Term  Repeatability  for  the  Green  Fabric 
Samples  on  the  DPSC  Spectro  Sensor 


Table  6a.  Comparison  of  Short-Term  Infrared  Repeatability  for  the  Tan  Fabric  Samples  between 
NLABS  and  DPSC:  Standard  Deviation  of  Starlight  Scope  Values  (Ns,Ls) 
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Table  7b.  Comparison  of  Long-Term  Infrared  Repeatability  for  the  Green  Fabric  Samples  between  NLABS  and  DPSC 


Table  8a.  Comparison  of  the  Short-  and  Long-Term  Infrared  Repeatability  for  the  Tan  Fabric 
Samples  on  the  NLABS  Spectro  Sensor:  Standard  Deviation  of 
Starlight  Scope  Values  (Ns.Ls) 


o  lo  o  CO  q 
(6  '  io  ‘ 

(D  (O 


OOCO^^CO*-COCM»-COCOCOO>C*)0>C»)»-^(»)<0 

cNqioqqq<-qroqioqtnqqqr^qioq 
tri  'iri  ‘<d  'in  tn  'iri  'id  in  ‘id  in 
concococoroMcocoM 


C0(NIOC*lO«-CMCM<D 

qqcNoqqqqq 

(d  '  (d  '  <d  in  ‘  <d 

q  (O  <o  (o  (o 


CO  CM  CO  r>« 

o  CM  o  a> 


m  S  ^ 


<«•  CO  CO  CM  00  CO 
CM  O  O  O  •-  O 


CM  o> 
CO  o  in 


CM  CM  CM 

O  CM  o 


Q  CO  O  CO  o 

^  cS  CO  o  CO 


^  CO  00 

(p  q  q 
in  '  (d 

CO  CO 


r~cor'»co't»-^Mg>5too5P^coq>5iococo5 

(jjo»-oinooboo>Oi-oo>ocMOoo«-o 


oocN^cococMCMCM»-cocococ»co»-ino'e-coin 
CMqinqqq»-qqqinqqqi>.pr';qqq 
in  'in  'cd  'in  in  'in  'cd  'in  cd  in 
cocococococococococo 


5tCMT-CNO’-^CMIOCOCMCMOOCOininCOCMOO'4- 

oqcMOcooa>oqqcMqqqcMqoqi-o 
cd  cd  cd  in  '<o  cd  cd  cd  'c^  cd 
cocococococococococo 


oo»-o)coo»-t-fMO>CMOcoocoin 

COOIOO^OCMOCOOCOOCOOCO 

!2  !S  SS  J2  JS  !S  SR  JR 
cocococococococo 


S(oS 

s  ■ 


OOCM«-CMO>OCOCOI^OO 

E  I  j2  2522*2”?::::® 

^HmmmcococoSKIoS 


Table  9a.  Comparison  of  the  Short-  and  Long-Term  Infrared  Repeatability  of  the  Tan  Fabric 
Samples  on  the  DPSC  Spectro  Sensor:  Standard  Deviation  of 
Starlight  Scope  Values  (Ns,Ls) 
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Table  9b.  Comparison  of  the  Short-  and  Long-Term  Infrared  Repeatability  for  the  Green  Fabric 

Samples  on  the  DPSC  Spectro  Smsor 
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Table  10a.  Comparison  of  Average  Short-Term  Values 
for  the  Two  Instruments 


Average  MCDM  Average  AE* 


Sample 

NLABS 

DPSC 

Tiles 

.04± 

.07± 

.09± 

.015 

.020 

.040 

Greens 

.05± 

.07± 

.08± 

.014 

.027 

.028 

Tans 

.04± 

.04± 

.06± 

.012 

.007 

.026 

Table  10b.  Comparison  of  Average  Long-Term  Values 
for  the  Two  Instruments 

Average  MCDM 

Average  AE* 

Sample 

NLABS 

DPSC 

Tiles 

.04± 

.09± 

.12± 

.010 

.034 

.028 

Greens 

.06± 

.06± 

.10± 

.010 

.008 

.020 

Tans 

.04± 

.04± 

.07± 

.011 

.006 

.026 

Table  10c. 

Comparison  of  Average  Long  and  Short-Term  Values 
for  the  NLABS  Instrument 

Average  MCDM 

Average  AE* 

Sample 

Short 

Long 

Tiles 

.04± 

.04± 

.04± 

.015 

.010 

.037 

Greens 

.05± 

.06± 

.05± 

.014 

.010 

.034 

Tans 

.04± 

.04± 

.03± 

.012 

.011 

.020 
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Table  lOd.  Comparison  of  Average  Long-  and  Short-Term  Values 
for  the  DPSC  Instrument 


Average  MCDM 

Average  AE* 

Sample 

Short 

Long 

Tiles 

.07± 

.og± 

.06± 

.020 

.034 

.044 

Greens 

.07± 

.06± 

.06± 

.027 

.008 

.041 

Tans 

.04± 

.04± 

.03± 

.007 

.006 

.016 
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